1. Field of the Invention
The present invention relates to a recognition unit and a recognizing and judging apparatus employing a plurality of recognition units for recognizing and judging an object according to various characteristic data of the object.
2. Description of the Prior Art
FIG. 1 is a block diagram indicating a conventional recognizing and judging apparatus, which comprises a template comparator 1 having a signal input terminal 2 and a signal output terminal 3, a plurality of templates 4-7 of input signal patterns prepared in advance, and a learning device 8 having a teacher signal input terminal 9.
A series of signals representative of an object to be recognized are inputted from the signal input terminal 2 to the template comparator 1. The template comparator 1 compares a pattern of the inputted signals with each of the templates 4-7 and outputs the number of a template having the smallest error as a recognition result from the signal output terminal 3. If the output result is incorrect, the number of a template to which the inputted signals should belong is inputted from the teacher signal input terminal 9 to the learning device 8, thereby modifying an appropriate template to improve the recognizing percentage.
In the conventional recognizing and judging apparatus as described above, because it is necessary to compare the inputted signals with all the templates, it takes very long time to make comparisons. Furthermore, because it is necessary to prepare the same number of templates as the kinds of patterns to be classified and recognized, a memory having a large capacity for storing all the templates is required. In addition, learning to be performed by modifying the appropriate template needs a lot of time.
A learning type recognizing and judging apparatus is known, for example, in "Learning Representations by Back-Propagating Errors" written by D. E. Rumelhart et al. and reported in Vol. 323, No. 9 (1986) of "Nature".
FIG. 2 is a block diagram indicating the structure of one of conventional learning type recognizing and judging apparatus, which comprises a plurality of input terminals 11 and 12, a plurality of output terminals 22, a learning circuit 27, a plurality of multi-input/output circuits 28-30, an output layer 31, and a hidden layer 32. As shown in FIG. 2, the plurality of multi-input/output circuits 28-30 are organized in a hierarchical manner in the learning type recognizing and judging apparatus, in which signals inputted from the input terminals 11 and 12 are processed and outputted from the output terminals 22. Of the hierarchically organized multi-input/output circuits 28-30, a layer containing multi-input/output circuits for outputting signals is called the "output layer" whereas a layer containing multi-input/output circuits other than those contained in the output layer is called the "hidden layer". The hidden layer may consist of either a single layer or a plurality of layers each including multi-input/output circuits.
FIG. 3 is a block diagram indicating the structure of another conventional learning type recognizing and judging apparatus, which comprises input terminals 41 and 42, a plurality of variable weight multipliers 43-48, a plurality of adders 49-51 each having a saturation input/output characteristic, an output terminal 52, a teacher signal generating section 53, an error calculating section 54, a steepest-descent-direction deciding section 55, a weight changing section 56, a learning circuit 57, a plurality of multi-input/output circuits 58-60, an output layer 61, and a hidden layer 62. As shown in FIG. 3, each of the multi-input/output circuits 58-60 includes two variable weight multipliers and one adder having the saturation input/output characteristic. More specifically, an output signal of the j-th multi-input/output circuit is expressed as: ##EQU1## where y[i] is an output signal of the i-th multi-input/output circuit of the preceding layer; w[i, j] is a weight to be applied when the output signal of the i-th multi-input/output circuit of the preceding layer is inputted to the j-th multi-input/output circuit; and fnc( ) is a function having a saturation characteristic and is expressed by, for example, the following Sigmoid function: ##EQU2##
FIG. 4 is a graph representative of the characteristic function of the adders 49-51 each having a saturation input/output characteristic expressed by the above fnc(). As described above, the learning type recognizing and judging apparatus has a structure in which multi-input/output circuits are hierarchically connected with each other. During learning, weights applied to the variable multipliers 43-48 are changed so that a desirable output signal (hereinafter referred to as a teacher signal) may be outputted in response to input signals. First, an error is found from the teacher signal and the output signal of the output layer in order to determine the amount of changes of the weights. ##EQU3## where y.sub.p [j] is an output signal of the j-th multi-input/output circuit of the output layer in response to the p-th input signal; t.sub.p [j] is a teacher signal with respect to y.sub.p [j]; ##EQU4## is the sum of all the teacher signals; ##EQU5## is the sum of all the multi-input/output circuits of the output layer; and w is a vector (hereinafter w is referred to as a weight vector) having a weight w[i, j] as a component. As indicated by Equation (2), the error E is expressed by the sum of squares of the differences between the teacher signals and the corresponding output signals of the output layer and is a function of the weight vector w. During the learning, the weights are changed and the differences between the teacher signals and actual output signals, namely the error, are minimized. The amount of changes of the weights is determined by ##EQU6## where .epsilon. is a positive constant called the "learning parameter"; .alpha. is a positive constant called the "momentum"; ##EQU7## is a vector of the partial derivative of the error E expressed by Equation (2) with respect to a weight w[i, j] and is called the "steepest descent direction"; and .DELTA.w' is a vector expression of the amount of changes of the weights in the previous learning.
FIG. 5 schematically depicts the structure of the learning circuit 57 of the conventional learning type recognizing and judging apparatus. The learning circuit 57 has an input terminal 63 for receiving output signals from the output layer, an input terminal 64 for receiving output signals from the hidden layer, an input terminal 65 for receiving input signals, an output terminal 66 for outputting weights to the output layer, and an output terminal 67 for outputting weights to the hidden layer. In this learning circuit 57, the teacher signal generating section 53 generates a teacher signal t.sub.p [j] (desirable output signal) in response to an input signal. The error calculating section 54 calculates the error E expressed by Equation (2) based on the teacher signal t.sub.p [j] and the output signal y.sub.p [j] of the output layer. The error calculating section 54 outputs a difference signal t.sub.p [j]-y.sub.p [j] between the teacher signal required for changing the weights and the output signal to the steepest-descent-direction deciding section 55. The steepest-descent-direction deciding section 55 finds the steepest descent direction of the error E in a space of weights, in which the weights are expressed in vector, based on output signals of the output layer, output signals of the hidden layer, input signals, and weights of the output layer. The steepest descent direction is found by: ##EQU8##
The right side of Equation (4) is a vector expression of the partial derivative of the error E with respect to the weight. The steepest-descent-direction deciding section 55 multiplies the steepest descent direction by the learning parameter and outputs the result to the weight changing section 56. The weight changing section 56 finds the amount of changes of the weights using Equation (3), thereby changing the weight to be multiplied in each of the variable weight multipliers 43-48. As described above, iteratively finding the amount of changes of the weights by the use of the steepest descent method can reduce the error. When the error becomes considerably small, the learning is terminated by reason that the output signal has reached a satisfactorily desirable value.
In the conventional recognizing and judging apparatus of the above-described type, the structure of a network cannot be altered from a fixed state initially defined or designed. Accordingly, since the learning and the recognition ability are determined by the initial design, the apparatus cannot provide an appropriate processing according to input data. Furthermore, any method of designing the network has not been established and actual designing fully depends upon trial and error based on experience and perception.